Skip to content

Add a configurable parameter to the number of backup attempts that are made.#1

Open
odysseaspenta wants to merge 3 commits intoorientechnologies:3.2.xfrom
odysseaspenta:3.2.x
Open

Add a configurable parameter to the number of backup attempts that are made.#1
odysseaspenta wants to merge 3 commits intoorientechnologies:3.2.xfrom
odysseaspenta:3.2.x

Conversation

@odysseaspenta
Copy link

Currently, if the backup fails for some reason, the server can get into a situation where it continuously schedules another backup to take place that can potentially fill up the backup log with entries. Here is example output from a server that is configured to perform backups every so often. When the backup fails, the server schedules another backup event to take place immediately. This behavior was present in 3.2.47.

2026-01-04T16:54:59 [INFO] Backup FULL_BACKUP in (ms):823 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:00 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:00 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:00 [INFO] Backup FULL_BACKUP in (ms):794 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:00 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:00 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:00 [INFO] Backup FULL_BACKUP in (ms):697 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:01 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:01 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:01 [INFO] Backup FULL_BACKUP in (ms):698 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:01 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:01 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:01 [INFO] Backup FULL_BACKUP in (ms):725 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:02 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:02 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:02 [INFO] Backup FULL_BACKUP in (ms):717 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:02 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:02 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:02 [INFO] Backup FULL_BACKUP in (ms):726 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:03 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:03 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:03 [INFO] Backup FULL_BACKUP in (ms):725 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:03 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:03 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:03 [INFO] Backup FULL_BACKUP in (ms):680 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:04 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:04 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:04 [INFO] Backup FULL_BACKUP in (ms):704 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:04 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:04 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:04 [INFO] Backup FULL_BACKUP in (ms):693 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:05 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:05 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:05 [INFO] Backup FULL_BACKUP in (ms):679 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:05 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:05 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:05 [INFO] Backup FULL_BACKUP in (ms):760 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:06 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:06 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:06 [INFO] Backup FULL_BACKUP in (ms):675 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:06 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:06 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:06 [INFO] Backup FULL_BACKUP in (ms):723 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T16:55:07 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)

Added support for limiting the number of times that a failed backup is attempted using a configurable parameter. This it to prevent the situation where the server gets in an endless loop of retrying to succesful make a backup that fails, filling up the backup log with entries. Here is the output of the same situation using the fixes suggested with this pull request. After 3 attempts, the server quits trying to make the backup.

2026-01-04T15:18:22 [INFO] Scheduled [FULL_BACKUP] task : a18c2be8-ab4e-485d-b0f8-bfe01bb94c54. Next execution will be Sun Jan 04 15:20:00 EST 2026 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:20:00 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:20:00 [WARNING] Backup failed for [oConfig-db]. Retry attempt 1/3 scheduled in 60 seconds (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:20:00 [INFO] Backup FULL_BACKUP in (ms):373 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:21:00 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:21:00 [WARNING] Backup failed for [oConfig-db]. Retry attempt 2/3 scheduled in 300 seconds (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:21:00 [INFO] Backup FULL_BACKUP in (ms):384 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:26:00 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:26:01 [WARNING] Backup failed for [oConfig-db]. Retry attempt 3/3 scheduled in 900 seconds (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:26:01 [INFO] Backup FULL_BACKUP in (ms):272 (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:41:01 [INFO] Backup started FULL_BACKUP (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:41:01 [WARNING] Backup failed for [oConfig-db] after 3 retries. Manual intervention required. Backup scheduling stopped. (com.orientechnologies.agent.services.backup.OBackupTask)
2026-01-04T15:41:01 [INFO] Backup FULL_BACKUP in (ms):758 (com.orientechnologies.agent.services.backup.OBackupTask)

…s attempted using a configurable parameter. This it to prevent the situation where the server gets in an endless loop of retrying to succesful make a backup that fails, filling up the backup log with entries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant